<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Using Kinect and other OpenNI compatible depth sensors</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d9/df8/tutorial_root.html">OpenCV Tutorials</a></li><li class="navelem"><a class="el" href="../../de/d3d/tutorial_table_of_content_app.html">Application utils (highgui, imgcodecs, videoio modules)</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Using Kinect and other OpenNI compatible depth sensors </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p><b>Prev Tutorial:</b> <a class="el" href="../../d7/d9e/tutorial_video_write.html">Creating a video with OpenCV</a></p>
<p><b>Next Tutorial:</b> <a class="el" href="../../d4/d65/tutorial_orbbec_astra.html">Using Orbbec Astra 3D cameras</a></p>
<p>Depth sensors compatible with OpenNI (Kinect, XtionPRO, ...) are supported through VideoCapture class. Depth map, BGR image and some other formats of output can be retrieved by using familiar interface of VideoCapture.</p>
<p>In order to use depth sensor with OpenCV you should do the following preliminary steps:</p>
<ol type="1">
<li>Install OpenNI library (from here <a href="http://www.openni.org/downloadfiles">http://www.openni.org/downloadfiles</a>) and PrimeSensor Module for OpenNI (from here <a href="https://github.com/avin2/SensorKinect">https://github.com/avin2/SensorKinect</a>). The installation should be done to default folders listed in the instructions of these products, e.g.: <div class="fragment"><div class="line">OpenNI:</div><div class="line">    Linux &amp; MacOSX:</div><div class="line">        Libs into: /usr/lib</div><div class="line">        Includes into: /usr/include/ni</div><div class="line">    Windows:</div><div class="line">        Libs into: c:/Program Files/OpenNI/Lib</div><div class="line">        Includes into: c:/Program Files/OpenNI/Include</div><div class="line">PrimeSensor Module:</div><div class="line">    Linux &amp; MacOSX:</div><div class="line">        Bins into: /usr/bin</div><div class="line">    Windows:</div><div class="line">        Bins into: c:/Program Files/Prime Sense/Sensor/Bin</div></div><!-- fragment --> If one or both products were installed to the other folders, the user should change corresponding CMake variables OPENNI_LIB_DIR, OPENNI_INCLUDE_DIR or/and OPENNI_PRIME_SENSOR_MODULE_BIN_DIR.</li>
<li>Configure OpenCV with OpenNI support by setting WITH_OPENNI flag in CMake. If OpenNI is found in install folders OpenCV will be built with OpenNI library (see a status OpenNI in CMake log) whereas PrimeSensor Modules can not be found (see a status OpenNI PrimeSensor Modules in CMake log). Without PrimeSensor module OpenCV will be successfully compiled with OpenNI library, but VideoCapture object will not grab data from Kinect sensor.</li>
<li>Build OpenCV.</li>
</ol>
<p>VideoCapture can retrieve the following data:</p>
<ol type="1">
<li>data given from depth generator:<ul>
<li>CAP_OPENNI_DEPTH_MAP - depth values in mm (CV_16UC1)</li>
<li>CAP_OPENNI_POINT_CLOUD_MAP - XYZ in meters (CV_32FC3)</li>
<li>CAP_OPENNI_DISPARITY_MAP - disparity in pixels (CV_8UC1)</li>
<li>CAP_OPENNI_DISPARITY_MAP_32F - disparity in pixels (CV_32FC1)</li>
<li>CAP_OPENNI_VALID_DEPTH_MASK - mask of valid pixels (not occluded, not shaded etc.) (CV_8UC1)</li>
</ul>
</li>
<li>data given from BGR image generator:<ul>
<li>CAP_OPENNI_BGR_IMAGE - color image (CV_8UC3)</li>
<li>CAP_OPENNI_GRAY_IMAGE - gray image (CV_8UC1)</li>
</ul>
</li>
</ol>
<p>In order to get depth map from depth sensor use VideoCapture::operator &gt;&gt;, e. g. : </p><div class="fragment"><div class="line">VideoCapture capture( <a class="code" href="../../d4/d15/group__videoio__flags__base.html#gga023786be1ee68a9105bf2e48c700294daa073098573ce0968cb7a79de1a5ce0ee">CAP_OPENNI</a> );</div><div class="line"><span class="keywordflow">for</span>(;;)</div><div class="line">{</div><div class="line">    Mat depthMap;</div><div class="line">    capture &gt;&gt; depthMap;</div><div class="line"></div><div class="line">    <span class="keywordflow">if</span>( <a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">waitKey</a>( 30 ) &gt;= 0 )</div><div class="line">        <span class="keywordflow">break</span>;</div><div class="line">}</div></div><!-- fragment --><p> For getting several data maps use VideoCapture::grab and VideoCapture::retrieve, e.g. : </p><div class="fragment"><div class="line">VideoCapture capture(0); <span class="comment">// or CAP_OPENNI</span></div><div class="line"><span class="keywordflow">for</span>(;;)</div><div class="line">{</div><div class="line">    Mat depthMap;</div><div class="line">    Mat bgrImage;</div><div class="line"></div><div class="line">    capture.grab();</div><div class="line"></div><div class="line">    capture.retrieve( depthMap, <a class="code" href="../../dc/dfc/group__videoio__flags__others.html#ggaeb44e5d729d41902496454c281d5adbfa2278fe906c8f7508e6e7a92f46888661">CAP_OPENNI_DEPTH_MAP</a> );</div><div class="line">    capture.retrieve( bgrImage, <a class="code" href="../../dc/dfc/group__videoio__flags__others.html#ggaeb44e5d729d41902496454c281d5adbfa467bac9001f40dea888f47ac55886c23">CAP_OPENNI_BGR_IMAGE</a> );</div><div class="line"></div><div class="line">    <span class="keywordflow">if</span>( <a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">waitKey</a>( 30 ) &gt;= 0 )</div><div class="line">        <span class="keywordflow">break</span>;</div><div class="line">}</div></div><!-- fragment --><p> For setting and getting some property of sensor` data generators use VideoCapture::set and VideoCapture::get methods respectively, e.g. : </p><div class="fragment"><div class="line">VideoCapture capture( <a class="code" href="../../d4/d15/group__videoio__flags__base.html#gga023786be1ee68a9105bf2e48c700294daa073098573ce0968cb7a79de1a5ce0ee">CAP_OPENNI</a> );</div><div class="line">capture.set( <a class="code" href="../../dc/dfc/group__videoio__flags__others.html#gga9817e2bad3908e54fcf16ffed015ff75a241b67030848f3bcf223d18ebfe75adc">CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE</a>, <a class="code" href="../../dc/dfc/group__videoio__flags__others.html#gga7879653f5149fe92a14bc0ba3430b9b4a08a35a2f66b2d645eb7817e852ff8ddc">CAP_OPENNI_VGA_30HZ</a> );</div><div class="line">cout &lt;&lt; <span class="stringliteral">&quot;FPS    &quot;</span> &lt;&lt; capture.get( <a class="code" href="../../dc/dfc/group__videoio__flags__others.html#gga694b077e6f87a82639f9e77fa950a6bba88571cf323ed746cfad77160272fe593">CAP_OPENNI_IMAGE_GENERATOR</a>+<a class="code" href="../../d4/d15/group__videoio__flags__base.html#ggaeb8dd9c89c10a5c63c139bf7c4f5704daf01bc92359d2abc9e6eeb5cbe36d9af2">CAP_PROP_FPS</a> ) &lt;&lt; endl;</div></div><!-- fragment --><p> Since two types of sensor's data generators are supported (image generator and depth generator), there are two flags that should be used to set/get property of the needed generator:</p>
<ul>
<li>CAP_OPENNI_IMAGE_GENERATOR &ndash; A flag for access to the image generator properties.</li>
<li>CAP_OPENNI_DEPTH_GENERATOR &ndash; A flag for access to the depth generator properties. This flag value is assumed by default if neither of the two possible values of the property is not set.</li>
</ul>
<p>Some depth sensors (for example XtionPRO) do not have image generator. In order to check it you can get CAP_OPENNI_IMAGE_GENERATOR_PRESENT property. </p><div class="fragment"><div class="line"><span class="keywordtype">bool</span> isImageGeneratorPresent = capture.get( CAP_PROP_OPENNI_IMAGE_GENERATOR_PRESENT ) != 0; <span class="comment">// or == 1</span></div></div><!-- fragment --><p> Flags specifying the needed generator type must be used in combination with particular generator property. The following properties of cameras available through OpenNI interfaces are supported:</p>
<ul>
<li>For image generator:<ul>
<li>CAP_PROP_OPENNI_OUTPUT_MODE &ndash; Three output modes are supported: CAP_OPENNI_VGA_30HZ used by default (image generator returns images in VGA resolution with 30 FPS), CAP_OPENNI_SXGA_15HZ (image generator returns images in SXGA resolution with 15 FPS) and CAP_OPENNI_SXGA_30HZ (image generator returns images in SXGA resolution with 30 FPS, the mode is supported by XtionPRO Live); depth generator's maps are always in VGA resolution.</li>
</ul>
</li>
<li>For depth generator:<ul>
<li><p class="startli">CAP_PROP_OPENNI_REGISTRATION &ndash; Flag that registers the remapping depth map to image map by changing depth generator's view point (if the flag is "on") or sets this view point to its normal one (if the flag is "off"). The registration process’s resulting images are pixel-aligned,which means that every pixel in the image is aligned to a pixel in the depth image.</p>
<p class="startli">Next properties are available for getting only:</p>
</li>
<li>CAP_PROP_OPENNI_FRAME_MAX_DEPTH &ndash; A maximum supported depth of Kinect in mm.</li>
<li>CAP_PROP_OPENNI_BASELINE &ndash; Baseline value in mm.</li>
<li>CAP_PROP_OPENNI_FOCAL_LENGTH &ndash; A focal length in pixels.</li>
<li>CAP_PROP_FRAME_WIDTH &ndash; Frame width in pixels.</li>
<li>CAP_PROP_FRAME_HEIGHT &ndash; Frame height in pixels.</li>
<li>CAP_PROP_FPS &ndash; Frame rate in FPS.</li>
</ul>
</li>
<li>Some typical flags combinations "generator type + property" are defined as single flags:<ul>
<li>CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE = CAP_OPENNI_IMAGE_GENERATOR + CAP_PROP_OPENNI_OUTPUT_MODE</li>
<li>CAP_OPENNI_DEPTH_GENERATOR_BASELINE = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_BASELINE</li>
<li>CAP_OPENNI_DEPTH_GENERATOR_FOCAL_LENGTH = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_FOCAL_LENGTH</li>
<li>CAP_OPENNI_DEPTH_GENERATOR_REGISTRATION = CAP_OPENNI_DEPTH_GENERATOR + CAP_PROP_OPENNI_REGISTRATION</li>
</ul>
</li>
</ul>
<p>For more information please refer to the example of usage <a href="https://github.com/opencv/opencv/tree/master/samples/cpp/videocapture_openni.cpp">videocapture_openni.cpp</a> in opencv/samples/cpp folder. </p>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:34 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
